Object detection system, object detection method, and program

ABSTRACT

An object detection system capable of tracking a moving object over a wide range, at a high processing speed, and with a high accuracy of recognition without increasing the number of image capturing apparatuses installed in one place is provided. A movement information acquisition unit acquires movement information, which is information regarding a moving object included in a first image captured by a first image capturing apparatus among a plurality of image capturing apparatuses arranged in positions different from one another. A search range determination unit determines a limited search range in a second image captured by a second image capturing apparatus among the plurality of image capturing apparatuses in accordance with the movement information. An object recognition unit performs recognition of the moving object for the search range in the second image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2020-142257, filed on Aug. 26, 2020, thedisclosure of which is incorporated herein in its entirety by reference.

BACKGROUND

The present disclosure relates to an object detection system, an objectdetection method, and a program, and in particular, to an objectdetection system, an object detection method, and a program fordetecting a moving object.

There have been techniques for detecting objects using an imagecapturing apparatus such as a camera. With regard to these techniques,Japanese Unexamined Patent Application Publication No. 2019-165501discloses a tracking system that tracks one target object by a pluralityof cameras. The tracking system disclosed in Japanese Unexamined PatentApplication Publication No. 2019-165501 includes a first camera thatcaptures a first video image, a second camera that captures a secondvideo image at an angle of view substantially the same as that of thefirst camera, and a processor. The processor performs first trackingprocessing for tracking a target object on the first video image andsecond tracking processing for tracking a target object on the secondvideo image. Further, the processor performs reliability calculationprocessing for calculating a first reliability, which is a reliability,of the target object, obtained by the first tracking processing, and asecond reliability, which is a reliability, of the target object,obtained by the second tracking processing. Further, the processorcompares the first reliability with the second reliability, specifiesthe tracking processing whose reliability is higher than that of theother tracking processing, and performs control processing forcontrolling, based on the result of the tracking processing that hasbeen specified, the other tracking processing.

SUMMARY

According to the technique disclosed in Japanese Unexamined PatentApplication Publication No. 2019-165501, a plurality of cameras (thefirst camera and the second camera) capture images with substantiallythe same angle of view. According to techniques of this kind, it isimpossible to track one target object for positions different from oneanother. Therefore, it is difficult to track a moving object over a widerange. Further, while the aim of the technique disclosed in JapaneseUnexamined Patent Application Publication No. 2019-165501 is to improvethe accuracy of the tracking by using a plurality of cameras in oneplace, it is possible that a plurality of cameras cannot be installed inone place. In this case, it is difficult to improve the speed of theobject recognition processing without reducing the accuracy of therecognition of the object.

The present disclosure provides an object detection system, an objectdetection method, and a program capable of tracking a moving object overa wide range, at a high processing speed, and with a high accuracy ofrecognition without increasing the number of image capturing apparatusesinstalled in one place.

An object detection system according to the present disclosure includes:a movement information acquisition unit configured to acquire movementinformation, which is information regarding a moving object included ina first image captured by a first image capturing apparatus among aplurality of image capturing apparatuses arranged in positions differentfrom one another; a search range determination unit configured todetermine a limited search range in a second image captured by a secondimage capturing apparatus among the plurality of image capturingapparatuses in accordance with the movement information; and an objectrecognition unit configured to perform recognition of the moving objectfor the limited search range in the second image.

Further, the object detection system according to the present disclosuremay further include the plurality of image capturing apparatuses.

Further, an object detection method according to the present disclosureincludes: acquiring movement information, which is information regardinga moving object included in a first image captured by a first imagecapturing apparatus among a plurality of image capturing apparatusesarranged in positions different from one another; determining a limitedsearch range in a second image captured by a second image capturingapparatus among the plurality of image capturing apparatuses inaccordance with the movement information; and performing recognition ofthe moving object for the limited search range in the second image.

Further, a program according to the present disclosure causes a computerto execute: acquiring movement information, which is informationregarding a moving object included in a first image captured by a firstimage capturing apparatus among a plurality of image capturingapparatuses arranged in positions different from one another;determining a limited search range in a second image captured by asecond image capturing apparatus among the plurality of image capturingapparatuses in accordance with the movement information; and performingrecognition of the moving object for the limited search range in thesecond image.

In the present disclosure, according to this configuration, a movingobject is detected only in a range where the moving object is highlylikely to appear. Therefore, the detection speed becomes higher and theaccuracy of the detection becomes higher than those in the case in whicha whole image is searched. Further, since the plurality of imagecapturing apparatuses are installed at positions different from oneanother, a moving object can be detected over a wide range. Therefore,the object detection system according to the present disclosure enablesa moving object to be tracked over a wide range, at a high processingspeed, and with a high accuracy of recognition without increasing thenumber of image capturing apparatuses installed in one place.

The search range determination unit may further determine the limitedsearch range when the movement information indicates that the movingobject is traveling toward an imaging region of the second imagecapturing apparatus.

According to the above configuration, the search range may be limitedwhen the moving object appears in the imaging region of the second imagecapturing apparatus. Therefore, it is possible to detect a moving objectmore definitely in a determined limited search range.

The search range determination unit may further determine apredetermined range in the second image that corresponds to a directionof an imaging region of the first image capturing apparatus to be thelimited search range.

According to the above configuration, a region where it is highly likelythat the moving object will appear in an angle of view of the imagecapturing apparatus is determined to be a limited search range.Therefore, it is possible to determine the search range moreappropriately.

The object recognition unit may further perform recognition of themoving object for the limited search range at a timing when the movingobject is estimated to reach an imaging region of the second imagecapturing apparatus.

According to the above configuration, the search range is limited at atiming when the moving object is estimated to reach the imaging regionof the second image capturing apparatus. It is therefore possible todetect a moving object more definitely.

The object detection system may further include a reliabilitycalculation unit configured to calculate, for each type of the movingobject, a reliability of recognition of the moving object using the typeand the reliability of the moving object obtained as a result of objectrecognition performed by the object recognition unit and the type andthe reliability of the moving object included in the movementinformation.

In this way, the amount of data regarding the reliability calculatedusing the plurality of pieces of object information is larger than thatregarding the reliability obtained in object recognition processingperformed by the object recognition unit. Therefore, the accuracy of thereliability calculated using the plurality of pieces of objectinformation becomes higher than that of the reliability obtained inobject recognition processing performed by the object recognition unit.Therefore, with the above configuration, it becomes possible to improvethe accuracy of the reliability.

The reliability calculation unit may further calculate, for each type ofthe moving object, a reliability by calculating an average of thereliability of the moving object obtained as a result of objectrecognition performed by the object recognition unit and the reliabilityof the moving object included in the movement information.

According to the above configuration, for each determined type of themoving object, a reliability in which a plurality of reliabilities aretaken into account is calculated. Therefore, it is possible to calculatea reliability more appropriately.

According to the present disclosure, it is possible to provide an objectdetection system, an object detection method, and a program capable oftracking a moving object over a wide range, at a high processing speed,and with a high accuracy of recognition without increasing the number ofimage capturing apparatuses installed in one place.

The above and other objects, features and advantages of the presentdisclosure will become more fully understood from the detaileddescription given hereinbelow and the accompanying drawings which aregiven by way of illustration only, and thus are not to be considered aslimiting the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing an object detection system according to afirst embodiment;

FIG. 2 is a diagram showing a hardware configuration of an objectdetection apparatus according to the first embodiment;

FIG. 3 is a diagram illustrating a state in which object detectionapparatuses according to the first embodiment are installed in a roadnetwork;

FIG. 4 is a diagram illustrating object information according to thefirst embodiment;

FIG. 5 is a diagram illustrating movement information according to thefirst embodiment;

FIG. 6 is a diagram illustrating an image captured by an image capturingapparatus according to the first embodiment;

FIG. 7 is a diagram illustrating an image captured by the imagecapturing apparatus according to the first embodiment;

FIG. 8 is a block diagram showing a configuration of an informationprocessing apparatus according to the first embodiment;

FIG. 9 is a flowchart showing an object detection method executed by anobject detection system according to the first embodiment;

FIG. 10 is a flowchart showing an object detection method executed bythe object detection system according to the first embodiment; and

FIG. 11 is a diagram for describing a specific example of processing ofthe object detection system according to the first embodiment.

DESCRIPTION OF EMBODIMENTS First Embodiment

Hereinafter, with reference to the drawings, embodiments of the presentdisclosure will be described. Note that the same symbols are assigned tosubstantially the same components.

FIG. 1 is a diagram showing an object detection system 1 according to afirst embodiment. The object detection system 1 includes a managementserver 50 and plurality of object detection apparatuses 100 (100A-100D).The management server 50 and the plurality of object detectionapparatuses 100 are connected to each other via a wired or wirelessnetwork 2 so that they can communicate with each other. Further, theplurality of object detection apparatuses 100A-100D may be connected toone another via the network 2 in such a way that they can communicatewith one another.

The management server 50 is, for example, a computer. The managementserver 50 may be, for example, a cloud server. The management server 50manages information generated in the object detection apparatuses 100(object information, movement information etc. that will be describedlater). The management server 50 may include a database (DB) that storesthis information.

The object detection apparatuses 100A-100D are arranged at positionsdifferent from one another. The object detection apparatuses 100A-100Dare arranged, for example, near intersections of a road. Further, theobject detection apparatuses 100A-100D may be arranged near the road atintervals of, for example, 50 m-100 m. The object detection apparatus100 is, for example, an infrastructure sensor including a camera, asensor or the like. The object detection apparatus 100 detects anobject, in particular, a moving object, which is a movable object. FIG.1 shows four object detection apparatuses 100A-100D. However, the numberof object detection apparatuses 100 is not limited to four and may beany number equal to or larger than two.

FIG. 2 is a diagram showing a hardware configuration of the objectdetection apparatus 100 according to the first embodiment. The objectdetection apparatus 100 includes an image capturing apparatus 101 and aninformation processing apparatus 110. That is, the object detectionapparatuses 100A-100D respectively include image capturing apparatuses101A-101D. The image capturing apparatus 101 may be physically separatedfrom the information processing apparatus 110. In this case, the imagecapturing apparatus 101 is connected to the information processingapparatus 110 using a wire or wirelessly so that they can communicatewith each other.

The image capturing apparatus 101 is, for example, a camera body. Theimage capturing apparatus 101 captures images of (photographs) apredetermined region (an imaging region) that corresponds to a positionwhere the image capturing apparatus 101 (the object detection apparatus100) is installed. The imaging region corresponds to an angle of view (afield of view; captured range) of the image capturing apparatus 101. Theimage capturing apparatus 101 can be regarded as an infrastructuresensor. The image capturing apparatus 101 may be a visible light camera,a three-dimensional camera (a point cloud camera) such as LiDAR (LightDetection and Ranging), or an infrared camera.

The information processing apparatus 110 performs object recognitionprocessing on an image captured by the image capturing apparatus 101.Then the information processing apparatus 110 generates objectinformation indicating a result of object recognition. The detailsthereof will be described later. In the following, the term “image” mayalso indicate “image data indicating an image”, which indicates a targetto be processed in information processing. Further, the image may be astill image or a moving image.

The information processing apparatus 110 includes, as a main hardwareconfiguration, a Central Processing Unit (CPU) 102, a Read Only Memory(ROM) 104, a Random Access Memory (RAM) 106, and an interface unit 108(IF; Interface). The CPU 102, the ROM 104, the RAM 106, and theinterface unit 108 are connected to one another via a data bus or thelike. The management server 50 may include a configuration that issubstantially the same as the hardware configuration of the informationprocessing apparatus 110 described above.

The CPU 102 has a function as an arithmetic apparatus that performscontrol processing, arithmetic processing and the like. The ROM 104includes a function of storing a control program, an arithmetic programand the like executed by the CPU 102. The RAM 106 includes a function oftemporarily storing process data and the like. The RAM 106 may include adatabase. Accordingly, the database may be implemented in theinformation processing apparatus 110. The interface unit 108 receives oroutputs signals from or to an external device using a wire orwirelessly. Further, the interface unit 108 accepts an operation forinputting data performed by a user and displays information for theuser.

FIG. 3 is a diagram illustrating a state in which the object detectionapparatuses 100 according to the first embodiment are arranged in a roadnetwork 4. An outline of this embodiment will be described withreference to FIG. 3 . The road network 4 includes a road 30. In theexample shown in FIG. 3 , the object detection apparatuses 100 (theimage capturing apparatuses 101) are arranged near intersections 40. Theimage capturing apparatuses 101 of the object detection apparatuses 100capture images of imaging regions 42 that correspond to theintersections 40. Then the information processing apparatuses 110 of theobject detection apparatuses 100 detect (recognize) a moving object thatis present in the imaging regions 42 using images captured by the imagecapturing apparatus 101. The information processing apparatuses 110perform object recognition using, for example, a machine learningalgorithm such as deep learning. The moving object, which is, forexample, a vehicle, may be a pedestrian. The vehicle includes, forexample, a bicycle, a motorcycle, an automobile, a bus, and a heavytruck.

For example, the object detection apparatus 100A (the image capturingapparatus 101A) is arranged, for example, near an intersection 40A. Thenthe image capturing apparatus 101A of the object detection apparatus100A captures images of an imaging region 42A that corresponds to theintersection 40A. Then the object detection apparatus 100A detects(recognizes) a moving object that is present in the imaging region 42Ausing the images captured by the image capturing apparatus 101A.

Further, the object detection apparatus 100B (the image capturingapparatus 101B) is arranged near an intersection 40B that is adjacent tothe intersection 40A. Then the image capturing apparatus 101B of theobject detection apparatus 100B captures images of an imaging region 42Bthat corresponds to the intersection 40B. The object detection apparatus100B then detects (recognizes) a moving object that is present in theimaging region 42B using the images captured by the image capturingapparatus 101B.

Further, the object detection apparatus 100C (the image capturingapparatus 101C) is arranged near an intersection 40C that is adjacent tothe intersection 40B. Then the image capturing apparatus 101C of theobject detection apparatus 100C captures images of an imaging region 42Cthat corresponds to the intersection 40C. Then the object detectionapparatus 100C detects (recognizes) a moving object that is present inthe imaging region 42C using the images captured by the image capturingapparatus 101C.

Further, the object detection apparatus 100D (the image capturingapparatus 101D) is arranged near an intersection 40D that is adjacent tothe intersection 40C. Then the image capturing apparatus 101D of theobject detection apparatus 100D captures images of an imaging region 42Dcorresponding to the intersection 40D. The object detection apparatus100D then detects (recognizes) a moving object that is present in theimaging region 42D using the images captured by the image capturingapparatus 101D.

Assume, for example, that a moving object S moves along the road 30 inthe order of the intersection 40A, the intersection 40B, theintersection 40C, and the intersection 40D. In this case, the movingobject S first enters the intersection 40A, passes through theintersection 40A, and travels toward the intersection 40B. When themoving object S traveling toward the intersection 40A enters the imagingregion 42A (an angle of view of the image capturing apparatus 101A),this moving object S is detected (recognized) by the object detectionapparatus 100A. When the moving object S further moves and passesthrough the intersection 40A, the moving object S exits the imagingregion 42A. In this case, the moving object S is no longer recognized bythe object detection apparatus 100A.

When the moving object S further moves toward the intersection 40B fromthe intersection 40A, the moving object S enters the imaging region 42Bfrom the direction of the intersection 40A. In this case, the movingobject S is detected (recognized) by the object detection apparatus100B. When the moving object S then exits the imaging region 42B, themoving object S is no longer recognized by the object detectionapparatus 100B.

Likewise, when the moving object S moves toward the intersection 40Cfrom the intersection 40B, the moving object S enters the imaging region42C from the direction of the intersection 40B. In this case, the movingobject S is detected (recognized) by the object detection apparatus100C. When the moving object S exits the imaging region 42C, the movingobject S is no longer recognized by the object detection apparatus 100C.

Likewise, when the moving object S further moves toward the intersection40D from the intersection 40C, the moving object S enters the imagingregion 42D from the direction of the intersection 40C. In this case, themoving object S is detected (recognized) by the object detectionapparatus 100D. When the moving object S exits the imaging region 42D,the moving object S is no longer recognized by the object detectionapparatus 100D.

Further, the information processing apparatus 110 of the objectdetection apparatus 100 generates object information, which isinformation regarding the recognized moving object. The objectinformation may be generated every time the moving object is recognized.Therefore, during a period in which one moving object continues to berecognized by the object detection apparatus 100, a plurality of piecesof object information may be generated for this moving object. Thenobject information regarding one moving object is generated in each ofthe plurality of object detection apparatuses 100A-100D.

FIG. 4 is a diagram illustrating the object information according to thefirst embodiment. The object information includes, for example, anobject ID, which is an identifier of the moving object, a type andreliability of the moving object, a recognition time, a recognitionposition (recognition latitude and recognition longitude), a recognitionorientation, a recognition speed, and a feature amount. Further, whenthe moving object is a vehicle and can be recognized by the informationprocessing apparatus 110, the object information may include the type ofthe vehicle, the color of the vehicle body, and the vehicle body number(number plate information).

The “recognition time” here indicates the time when object recognitionprocessing has been performed for the corresponding moving object. The“recognition position” indicates the geographical position of the movingobject when the object recognition processing has been performed. The“recognition position” may be calculated, for example, from the positionof pixels that correspond to the moving object in the image captured bythe image capturing apparatus 101. The recognition position may becalculated by, for example, associating a pixel position in an imagecaptured by the image capturing apparatus 101 with a position on theroad in advance. The “recognition orientation” indicates the movingdirection (e.g., North, South, East, and West) of the moving object whenthe object recognition processing has been performed. The recognitionorientation may be calculated by, for example, associating a directionin the image captured by the image capturing apparatus 101 with theorientation in advance and detecting in which direction on the image themoving object has moved. The “recognition speed” indicates the movingspeed of the moving object when the object recognition processing hasbeen performed. The recognition speed may be calculated, for example, byassociating a pixel position in the image captured by the imagecapturing apparatus 101 with a position on the road in advance andcalculating the difference between the positions of the moving object onthe road per unit time.

Further, the “type” is a category (class) of the moving objectdetermined in the object recognition processing. The type of the movingobject is, for example, a heavy truck, a bus, an automobile, amotorcycle, a bicycle, or a pedestrian. Further, the “reliability”indicates the likelihood that the type of the moving object is likely tobe correct (class probability; reliability score), which is determinedin the object recognition processing. In the object information, thenumber of sets of the type and the reliability is not limited to one.For example, the type and the reliability may be expressed, for example,as “heavy truck: 0.3, bus: 0.7”. The “feature amount” indicates featuresof the moving object extracted from the image captured by the imagecapturing apparatus 101. The feature amount may be extracted in theobject recognition processing and may be used to determine the type ofthe object. Further, “the type of the vehicle” may be determined using,for example, a recognition dictionary in which the feature amount isassociated with the type of the vehicle. Further, the vehicle bodynumber may be extracted using, for example, Optical CharacterRecognition (OCR).

Further, when the moving object moves and exits the correspondingimaging region 42, the information processing apparatus 110 generatesmovement information (passage information) including object informationgenerated just before the moving object exits the imaging region 42 andthe installation position information regarding the position where theimage capturing apparatus 101 is installed. Therefore, it can be saidthat the movement information is information regarding the moving objectthat has moved away from the imaging region 42. The movement informationmay be generated, for example, at predetermined time intervals.

FIG. 5 is a diagram illustrating the movement information according tothe first embodiment. The movement information includes installationposition information, current situation information, and objectinformation. The installation position information is informationregarding the place where the object detection apparatus 100 (the imagecapturing apparatus 101) is installed. In the example shown in FIG. 5 ,the installation position information is information regarding theintersection 40. When, for example, the object detection apparatus 100Agenerates the movement information, the installation positioninformation is information regarding the intersection 40A. Theinstallation position information includes, for example, theidentification number of the intersection 40 and installation positioninformation (installation position latitude and installation positionlongitude) indicating the position of the intersection 40. Theidentification number may be an identification number of the objectdetection apparatus 100 or the image capturing apparatus 101. Further,the installation position information may be positional information ofthe image capturing apparatus 101 or the imaging region 42.

The current situation information indicates the current situation in theintersection 40. When, for example, the object detection apparatus 100Agenerates movement information, the current situation informationindicates the situation in the intersection 40A. The current situationinformation may indicate the current time, the operating state, thedegree of congestion in the intersection 40, and the number of passingobjects, which is the number of objects passing through the intersection40.

The object information corresponds to the object information illustratedin FIG. 4 . The object information included in the movement informationrelates to a moving object that is no longer recognized since it hasexited the angle of view (the imaging region 42) of the image capturingapparatus 101. Specifically, the object information included in themovement information corresponds to the object information generatedlast before the corresponding moving object exits the angle of view (theimaging region 42) of the image capturing apparatus 101. In other words,the object information included in the movement information is objectinformation generated just before the corresponding moving object exitsthe angle of view (the imaging region 42) of the image capturingapparatus 101. Therefore, the recognition time, the recognitionposition, the recognition orientation, and the recognition speed may bea final recognition time, a final recognition position, a finalrecognition orientation, and a final recognition speed, respectively.

The object detection apparatus 100 (the information processing apparatus110) may transmit the generated movement information to an objectdetection apparatus 100 arranged near an intersection 40 adjacent to theintersection 40 corresponding to itself via the network 2. In this case,the object detection apparatus 100 arranged near the adjacentintersection 40 described above may acquire the movement informationfrom the object detection apparatus 100. Further, the informationprocessing apparatus 110 may store the generated movement information ina database of the management server 50 via the network 2. In this case,the object detection apparatus 100 arranged near the adjacentintersection 40 described above may acquire (receive) the movementinformation from the management server 50. That is, the object detectionapparatus 100 (e.g., the object detection apparatus 100B) acquiresmovement information regarding the moving object included in an image(first image) captured by one image capturing apparatus 101 (e.g., theimage capturing apparatus 101A; the first image capturing apparatus)among a plurality of image capturing apparatuses 101.

Then, the object detection apparatus 100 that has acquired the movementinformation determines a limited search range in an image (second image)captured by an image capturing apparatus 101 (second image capturingapparatus) among a plurality of image capturing apparatuses 101 inaccordance with the above movement information. The object detectionapparatus 100B that has acquired the movement information determines,for example, a limited search range in the image (second image) capturedby the image capturing apparatus 101B (the second image capturingapparatus) in accordance with the above movement information. Then, thisobject detection apparatus 100 (e.g., the object detection apparatus100B) performs object recognition of the moving object for the limitedsearch range. Limiting the search range in the object recognition inthis manner is referred to as a search limitation.

Assume, for example, that the moving object S passes through theintersection 40A and travels toward the intersection 40B. In this case,the object detection apparatus 100A generates object informationregarding the moving object S when the moving object S is present in theimaging region 42A. When the moving object S exits from the imagingregion 42A, the object detection apparatus 100A generates movementinformation regarding the moving object S. In this case, the objectdetection apparatus 100A may transmit the movement information to theadjacent object detection apparatus 100 (the object detection apparatus100B and an object detection apparatus (not shown) which is opposite tothe object detection apparatus 100B). Further, the object detectionapparatus 100A may transmit the movement information to the managementserver 50.

On the other hand, the object detection apparatus 100B acquires themovement information generated by the object detection apparatus 100Afrom the object detection apparatus 100A or the management server 50. Atthis time, the object detection apparatus 100B estimates that the movingobject S will reach the imaging region 42B from the direction of theintersection 40A (the imaging region 42A) at time tb. That is, theobject detection apparatus 100B estimates the timing when the movingobject S reaches the imaging region 42B from the direction of theintersection 40A (the imaging region 42A). At this time, the objectdetection apparatus 100B limits the search range of the moving object toa predetermined region in the direction of the intersection 40A in theimage captured by the image capturing apparatus 101B, at time tb. Inthis manner, the object detection apparatus 100B determines a limitedsearch range in the image (second image) captured by the image capturingapparatus 101B. Then, the object detection apparatus 100B performsobject recognition of the moving object S in the limited search range,at time tb.

FIGS. 6 and 7 are diagrams each illustrating an image captured by theimage capturing apparatus 101 according to the first embodiment. FIGS. 6and 7 each illustrate an image ImB at the intersection 40B captured bythe image capturing apparatus 101B. In the image ImB, the back sidecorresponds to the direction of the intersection 40A and the front sidecorresponds to the direction of the intersection 40C. Then the objectdetection apparatus 100B sets a limited search range 60 as illustratedin FIG. 7 when the moving object S reaches the angle of view (theimaging region 42B) of the image capturing apparatus 101B from thedirection of the intersection 40A. Then the object detection apparatus100B performs object recognition in the limited search range 60illustrated in FIG. 7 at a timing when the moving object S appears inthe angle of view (the imaging region 42B) of the image capturingapparatus 101B from the direction of the intersection 40A.

Now, effects of the object detection system 1 according to the firstembodiment will be described along with the problem according to theabove-described technique disclosed in Japanese Unexamined PatentApplication Publication No. 2019-165501. When, for example, a movingobject is recognized using a visible light camera, the objectrecognition is performed based on a feature amount of a pixel data groupfor each still image frame forming a video image in principle.Therefore, the higher the image quality becomes, the larger the range tobe recognized becomes and the longer the processing time for each stillimage frame becomes. Further, due to characteristics of a visible lightcamera, the position and the size of the object on a projected stillimage frame vary greatly depending on the position of the object to berecognized (the distance or the angle from the camera). It is thereforedifficult to improve the accuracy of recognition. When it is determinedthat the recognition degree (reliability score) is low in the objectrecognition, it is assumed that there is no object (missedrecognitions).

In the aforementioned technique disclosed in Japanese Unexamined PatentApplication Publication No. 2019-165501, when one target object istracked by a plurality of cameras installed in one place, the result ofthe tracking in one of the plurality of cameras whose reliability ishigher than those of the other ones is employed, thereby improving theaccuracy of the tracking. Therefore, detection and tracking by aplurality of cameras need to be performed at substantially the sametime, and the technique disclosed in Japanese Unexamined PatentApplication Publication No. 2019-165501 cannot be applied to tracking ofa moving object at different capturing times, that is, tracking of themoving object by cameras located away from one another. Therefore, inthe technique disclosed in Japanese Unexamined Patent ApplicationPublication No. 2019-165501, it is difficult to track an object over awide range.

Further, when object recognition is performed using a video image, thenumber of images whose object needs to be recognized per unit timebecomes enormous. Furthermore, as described above, when the search range(the size of this range) increases and the number of types of objects tobe detected increases as the quality of the image becomes higher, theobject recognition processing may become processing with an extremelyheavy load.

On the other hand, when the moving object is tracked over a wide range,the object recognition processing is preferably performed by edgecomputers (or computers on a network aggregated to some extent)installed at places where cameras are installed in view of the problemof a network cost (load). By executing the object recognition processingin a large number of places, it is possible to grasp the movement stateof the object or grasp the traffic volume over a wide range. While it isrequired to arrange a large number of edge computers in order to executethe object recognition processing in a large number of places, it isdifficult to improve hardware specifications of each edge computer inview of the cost and the like. It is therefore required to reduce theamount of information to be processed and perform object recognitionduring an appropriate process time while preventing the accuracy of theobject recognition from being reduced in the object recognitionprocessing without improving the hardware specifications.

On the other hand, the object detection system 1 according to the firstembodiment acquires movement information, which is information regardingthe moving object included in, for example, the first image captured bythe image capturing apparatus 101A (first image capturing apparatus)among the plurality of image capturing apparatuses 101 arranged inpositions different from one another. Further, the object detectionsystem 1 determines the limited search range in, for example, the secondimage captured by the image capturing apparatus 101B among the pluralityof image capturing apparatuses 101 in accordance with the acquiredmovement information. Then the object detection system 1 performsrecognition of the moving object for the limited search range in thesecond image. According to this configuration, the object detectionsystem 1 according to the first embodiment detects a moving object onlyfor a range where the moving object is highly likely to appear.Therefore, the detection speed becomes higher and the accuracy of thedetection becomes higher than those in the case in which the whole imageis searched. Further, since the plurality of image capturing apparatuses101 are arranged at positions different from one another, a movingobject can be detected over a wide range. Therefore, the objectdetection system 1 according to the first embodiment is able to track amoving object over a wide range, at a high processing speed, and with ahigh accuracy of recognition without increasing the number of imagecapturing apparatuses installed in one place.

Next, a configuration of the information processing apparatus 110 willbe described.

FIG. 8 is a block diagram showing a configuration of the informationprocessing apparatus 110 according to the first embodiment. Theinformation processing apparatus 110 includes a movement informationacquisition unit 112, a search limitation determination unit 114, and asearch range determination unit 116. Further, the information processingapparatus 110 includes an image acquisition unit 122, an objectrecognition unit 124, an object information extraction unit 126, areliability calculation unit 128, a movement information generation unit130, and a movement information transmission unit 132. These componentsmay be implemented, for example, by the CPU 102 executing a programstored in the ROM 104. Further, a necessary program may be stored in anarbitrary non-volatile storage medium and installed as necessary,whereby each component may be implemented. Each component is not limitedto be implemented by software as described above and may be implementedby hardware such as some sort of circuit element. Further, one or moreof the above components may be implemented by physically separatehardware. For example, one or more of the above components may beprovided by the management server 50.

In the following description of the components, the object detectionapparatus 100 (the image capturing apparatus 101) that corresponds tothe information processing apparatus 110 including each component (e.g.,the movement information acquisition unit 112) may be simply referred toas an “own object detection apparatus 100 (own image capturing apparatus101)”. Further, the intersection 40 (the imaging region 42) thatcorresponds to the own object detection apparatus 100 may be simplyreferred to as “the corresponding intersection 40 (the correspondingimaging region 42)” or “own intersection 40 (own imaging region 42)”.Further, an object detection apparatus 100 (the image capturingapparatus 101) that corresponds to an intersection 40 that is adjacentto the intersection 40 that corresponds to the own object detectionapparatus 100 may be simply referred to as an “adjacent object detectionapparatus 100 (adjacent image capturing apparatus 101)” or “objectdetection apparatus 100 (image capturing apparatus 101) adjacent to theown object detection apparatus 100”. Further, the imaging region 42 (theintersection 40) that corresponds to the adjacent object detectionapparatus 100 may be simply referred to as an “adjacent imaging region42 (adjacent intersection 40)”.

The movement information acquisition unit 112 is configured to acquirethe aforementioned movement information. The movement informationacquisition unit 112 may receive, for example, the movement informationfrom an object detection apparatus 100 that corresponds to anintersection 40 that is adjacent to the intersection 40 that correspondsto the own object detection apparatus 100 via the network 2. In thiscase, the movement information acquisition unit 112 may store theacquired movement information in the database of the management server50. Further, when the movement information is stored in the database ofthe management server 50, the movement information acquisition unit 112may receive the movement information from the management server 50 viathe network 2.

The movement information acquisition unit 112 may acquire, for example,at a timing when the movement information that corresponds to anintersection 40 that is adjacent to the intersection 40 that correspondsto the own object detection apparatus 100 has been generated, thismovement information. When, for example, movement information generatedin one object detection apparatus 100 has been transmitted from thisobject detection apparatus 100, the management server 50 may transmitthe movement information to an object detection apparatus 100 thatcorrespond to an intersection 40 that is adjacent to the intersection 40that corresponds to the above movement information. Accordingly, themovement information acquisition unit 112 is able to acquire, at thetiming when the movement information that corresponds to an intersection40 that is adjacent to the intersection 40 that corresponds to the ownobject detection apparatus 100 has been generated, this movementinformation. Alternatively, the movement information acquisition unit112 may acquire the movement information at predetermined intervals.

The search limitation determination unit 114 is configured to determinewhether or not the acquired movement information indicates that themoving object is traveling toward the own imaging region 42. Then thesearch limitation determination unit 114 is configured to determine thata search limitation will be imposed when it is determined that theacquired movement information indicates that the moving object istraveling toward the own imaging region 42.

Specifically, the search limitation determination unit 114 determineswhether or not object recognition will be performed in a limited searchrange in the own object detection apparatus 100. That is, the searchlimitation determination unit 114 determines whether or not a searchlimitation will be imposed when object recognition is performed in thefuture in the own object detection apparatus 100. Specifically, thesearch limitation determination unit 114 determines whether or not themoving object is traveling toward the intersection 40 (the imagingregion 42) that corresponds to the own object detection apparatus 100using the movement information generated in the adjacent objectdetection apparatus 100. For example, the search limitationdetermination unit 114 determines whether or not one moving object istraveling toward the intersection 40 that corresponds to the own objectdetection apparatus 100 from the final recognition orientation and thefinal recognition position (the final recognition latitude and the finalrecognition longitude) of the object information included in themovement information.

When it has been determined that one moving object is traveling towardthe imaging region 42 that corresponds to the own object detectionapparatus 100, the search limitation determination unit 114 estimatesthe timing when this moving object reaches the imaging region 42 thatcorresponds to the own object detection apparatus 100 (arrival time).Specifically, the search limitation determination unit 114 calculatesthe distance between the final recognition position of the objectinformation included in the movement information and the position of theown imaging region 42. Then the search limitation determination unit 114calculates the timing when the moving object reaches the imaging region42 that corresponds to the own object detection apparatus 100 (arrivaltime) from the final recognition time and the final recognition speed ofthe object information included in the movement information.Specifically, the search limitation determination unit 114 calculatesthe time required to travel the calculated distance at the finalrecognition speed. Then the search limitation determination unit 114adds the calculated time to the final recognition time, therebycalculating the timing when the moving object reaches the imaging region42 that corresponds to the own object detection apparatus 100 (arrivaltime).

In this case, the search limitation determination unit 114 may determinethat the moving object will appear in the own imaging region 42 at theestimated timing from the direction of the intersection 40 thatcorresponds to the object detection apparatus 100 that has generated themovement information. Then the search limitation determination unit 114determines that a search limitation will be imposed at the estimatedtiming. Further, the search limitation may be performed for apredetermined period of time including the estimated timing. The searchlimitation determination unit 114 may store the result of thedetermination in the database of the management server 50.Alternatively, the search limitation determination unit 114 may storethe result of the determination in the database implemented in theinformation processing apparatus 110.

The search range determination unit 116 is configured to determine thelimited search range in the image captured by the image capturingapparatus 101 of the own object detection apparatus 100 in accordancewith the acquired movement information. Specifically, the search rangedetermination unit 116 determines the search range when the acquiredmovement information indicates that the moving object is travelingtoward the own imaging region 42. More specifically, the search rangedetermination unit 116 determines the limited search range in an imagecaptured by the own object detection apparatus 100 (the image capturingapparatus 101) when the search limitation is imposed at the timingestimated by the search limitation determination unit 114. Then thesearch range determination unit 116 determines a predetermined range ofthe direction of the imaging region 42 of the adjacent object detectionapparatus 100 that has generated the movement information in the imagecaptured by the own image capturing apparatus 101 as a search range.

For example, like the limited search range 60 shown in FIG. 7 , thesearch range determination unit 116 determines a predetermined regionincluding at least a part of the road that leads to the adjacent imagingregion 42 (the intersection 40) in the image captured by the own imagecapturing apparatus 101 to be a search range. The search rangedetermination unit 116 may store the result of the determination(determined search range) in the database of the management server 50.Alternatively, the search range determination unit 116 may store theresult of the determination (determined search range) in a databaseimplemented by the information processing apparatus 110.

The image acquisition unit 122 is configured to acquire images from theimage capturing apparatus 101. Specifically, the image acquisition unit122 acquires images captured by the image capturing apparatus 101 of theown object detection apparatus 100. The acquisition of images may beexecuted for each timing when the image capturing apparatus 101 capturesan image.

The object recognition unit 124 is configured to perform objectrecognition processing on the acquired image. Specifically, the objectrecognition unit 124 detects (recognizes) a moving object from theacquired image by, for example, a machine learning algorithm such asdeep learning. The object recognition unit 124 extracts an object fromthe acquired image and calculates, for each extracted object, the type,the position, the speed, the direction in which it moves and the like.The object recognition unit 124 then generates object information (FIG.4 ) using the result of the recognition. Then the object recognitionunit 124 may transmit the generated object information to the managementserver 50, thereby storing this information in the database of themanagement server 50. Alternatively, the object recognition unit 124 maystore the object information in a database implemented in theinformation processing apparatus 110.

At this stage, the object ID (FIG. 4 ) in the object information may beset arbitrarily. That is, even when a moving object recognized before asa result of object recognition performed by the adjacent objectdetection apparatus 100 is the same as the moving object recognized bythe own object detection apparatus 100, an arbitrary object ID may begiven.

When there is a search limitation, the object recognition unit 124 isconfigured to perform object recognition for the limited search range.Specifically, the object recognition unit 124 performs objectrecognition for the search range determined by the search rangedetermination unit 116 at a timing of the search limitation estimated bythe search limitation determination unit 114. When a search limitationis imposed, the movement information is generated after processing inthe object information extraction unit 126 and the reliabilitycalculation unit 128 that will be described later is performed. On theother hand, when there is no search limitation, the object recognitionunit 124 is configured to perform object recognition for the wholeimage. When there is no search limitation, movement information isgenerated without performing processing in the object informationextraction unit 126 and the reliability calculation unit 128 that willbe described later. The object recognition unit 124 may determinewhether or not there is a search limitation by accessing the database ofthe management server 50 or the database implemented in the informationprocessing apparatus 110.

As described above, the object information extraction unit 126 performsprocessing when a search limitation is imposed. The object informationextraction unit 126 is configured to extract object informationregarding a moving object the same as the moving object recognized bythe object recognition processing performed by the object recognitionunit 124 from the database. Specifically, the object informationextraction unit 126 extracts information on an object including afeature amount whose similarity to the feature amount of the recognizedmoving object is equal to or larger than a predetermined value as objectinformation regarding a moving object the same as the moving objectrecognized by object recognition. That is, in this case, the objectinformation extraction unit 126 determines a moving object whose featureamount is similar to that of the recognized moving object as a movingobject the same as the recognized moving object. Alternatively, when thetype of the vehicle, the color of the vehicle body, and the vehicle bodynumber are included in the object information, the object informationextraction unit 126 may extract object information where theseinformation (the type of the vehicle, etc.) coincide with that includedin object information regarding the recognized moving object. Note thatthe object information extraction unit 126 may rewrite the object ID ofthe object information generated for the recognized moving object intoan object ID of the extracted object information. Accordingly, oneobject ID may be associated with the object information of one movingobject.

When a search limitation is imposed, it is highly likely that the movingobject recognized in the limited search range has been also recognizedby an adjacent object detection apparatus 100. Therefore, it is highlylikely that object information regarding a moving object the same as theabove moving object has already been stored in a database. On the otherhand, when a search limitation is not imposed, it is possible that amoving object that is the same as the recognized moving object may nothave been recognized by the adjacent object detection apparatus 100.Therefore, the object information regarding a moving object the same asthe above moving object may not be stored in the database. Therefore,when there is search limitation, the object information extraction unit126 extracts object information regarding a moving object the same asthe recognized moving object from the database. The object informationextraction unit 126 may perform the aforementioned processing even whena search limitation is not imposed. However, as described above, sinceit is possible that object information regarding a moving object thesame as the recognized moving object may not be stored in a database, anenormous amount of time may be required to perform extractionprocessing. That is, it may take time before it is determined thatobject information regarding a moving object the same as the recognizedmoving object is not stored in a database.

As described above, the reliability calculation unit 128 performsprocessing when a search limitation is imposed. The reliabilitycalculation unit 128 is configured to re-calculate the reliability ofthe type of the recognized moving object using the object informationextracted by the object information extraction unit 126. Specifically,the reliability calculation unit 128 re-calculates, for each recognizedmoving object, the reliability of the type of the moving object usingthe reliability in object information of a moving object determined tobe the same as the above moving object and the reliability in objectinformation regarding the recognized moving object. In other words, thereliability calculation unit 128 updates the reliability regarding therecognized moving object using the reliability regarding a moving objectdetermined to be the same. In other words, the reliability calculationunit 128 calculates the reliability of the recognition of the movingobject for each type using the type and the reliability of the movingobject obtained as a result of object recognition performed by theobject recognition unit 124 and the type and the reliability of themoving object included in the acquired movement information.

More specifically, the reliability calculation unit 128 re-calculatesthe reliability of the moving object S using the reliability of the typeobtained by the own object detection apparatus 100 and the reliabilityof the type obtained by the adjacent object detection apparatus 100.This re-calculated reliability is called an “integrated reliability”. Itis assumed, for example, regarding the moving object S, that the objectinformation generated by the adjacent object detection apparatus 100indicates that the reliability of a type A1 (e.g., the “heavy truck”) isBa1 and the reliability of a type A2 (e.g., the “bus”) is Ba2. It isfurther assumed that the object information generated by the own objectdetection apparatus 100 indicates that the reliability of the type A1 isBb1 and the reliability of the type A2 is Bb2. In this case, thereliability calculation unit 128 calculates, regarding the type A1, theintegrated reliability by calculating the average of the reliability Ba1and the reliability Bb1. Likewise, the reliability calculation unit 128calculates, regarding the type A2, the integrated reliability bycalculating the average of the reliability Ba2 and the reliability Bb2.For example, the reliability calculation unit 128 may calculate theupdated reliability (integrated reliability) by calculating, for eachtype, the (weighted) harmonic mean of reliabilities. In the aboveexample, the reliability calculation unit 128 calculates, regarding thetype A1, the integrated reliability by calculating the weighted harmonicmean of the reliability Ba1 and the reliability Bb1. Likewise, thereliability calculation unit 128 calculates, regarding the type A2, theintegrated reliability by calculating the weighted harmonic mean of thereliability Ba2 and the reliability Bb2.

The movement information generation unit 130 is configured to generatemovement information (FIG. 5 ) including object information regarding amoving object that is no longer recognized. Specifically, the movementinformation generation unit 130 generates movement information includingobject information regarding a moving object that is no longerrecognized by the object recognition unit 124 since it has exited theown intersection 40 (the imaging region 42) among all the moving objectsthat have already been recognized. Note that the movement informationgeneration unit 130 may generate movement information at predeterminedtime intervals.

For example, the movement information generation unit 130 extracts, foreach object ID, the latest object information among those whose“recognition time” is earlier than the current time by a predeterminedperiod of time or more from the database. Accordingly, the movementinformation generation unit 130 is able to extract the latest objectinformation regarding a moving object that has exited the own imagingregion 42. Then the movement information generation unit 130 sets thisextracted object information as “object information” in the movementinformation shown in FIG. 5 . The movement information generation unit130 further generates installation position information regarding theown object detection apparatus 100 and the current situationinformation. Accordingly, the movement information generation unit 130generates the movement information. Note that the movement informationgeneration unit 130 may store the movement information in the databaseimplemented in the information processing apparatus 110.

The movement information transmission unit 132 is configured to transmitthe movement information generated by the movement informationgeneration unit 130. The movement information acquisition unit 112 maytransmit the movement information to the management server 50 via thenetwork 2, thereby storing the movement information in the database ofthe management server 50. Further, the movement information transmissionunit 132 may transmit the movement information to an object detectionapparatus 100 adjacent to the own object detection apparatus 100 via thenetwork 2. In this case, the movement information transmission unit 132may transmit the movement information to an object detection apparatus100 that corresponds to an intersection 40 which is in the direction inwhich the moving object travels (final recognition orientation).

FIGS. 9 and 10 are flowcharts each showing the object detection methodexecuted by the object detection system 1 according to the firstembodiment. The processing shown in FIG. 9 may be executed, for example,at predetermined time intervals. As described above, the movementinformation acquisition unit 112 acquires the movement information (StepS102). Specifically, the movement information acquisition unit 112acquires movement information regarding a moving object included in theimage (first image) captured by the adjacent image capturing apparatus101 (the first image capturing apparatus).

As described above, the search limitation determination unit 114determines whether or not to impose a search limitation when objectrecognition is performed in the future by the own object detectionapparatus 100 using the movement information (Step S104). That is, thesearch limitation determination unit 114 determines whether or not themoving object regarding the movement information is traveling toward theown imaging region 42. At this time, as described above, the searchlimitation determination unit 114 determines the timing when a searchlimitation is imposed. Then the search limitation determination unit 114stores the result of the determination in the database.

When it is determined that the moving object regarding the movementinformation is not traveling toward the own imaging region 42, thesearch limitation determination unit 114 determines that a searchlimitation will not be imposed (NO in S104). In this case, thesubsequent processing S106 is omitted. On the other hand, when it isdetermined that the moving object regarding the movement information istraveling toward the own imaging region 42, the search limitationdetermination unit 114 determines that a search limitation will beimposed (YES in S104). In this case, as described above, the searchrange determination unit 116 determines the limited search range in theimage captured by the image capturing apparatus 101 of the own objectdetection apparatus 100 in accordance with the acquired movementinformation (Step S106). Then the search range determination unit 116stores the determined search range in the database.

In other words, the search range determination unit 116 determines thesearch range when the movement information indicates that the movingobject is traveling toward the own imaging region 42 (imaging region ofthe second image capturing apparatus). Accordingly, the search range maybe limited when the moving object appears in the own imaging region 42.Therefore, it is possible to detect a moving object more definitely in adetermined search range.

Further, as described above, the search range determination unit 116determines a predetermined range in the direction of the imaging region42 of the adjacent image capturing apparatus 101 (the first imagecapturing apparatus) in the image captured by the own image capturingapparatus 101 (the second image) to be a search range. Accordingly, arange where a moving object is highly likely to appear in the angle ofview of the image capturing apparatus 101 is determined to be a searchrange. Therefore, it is possible to determine the search range moreappropriately.

The processing shown in FIG. 10 may be executed, for example, every timethe own image capturing apparatus 101 captures an image. As describedabove, the image acquisition unit 122 acquires an image from the ownimage capturing apparatus 101 (Step S112). The object recognition unit124 determines whether or not there is a search limitation (Step S114).Specifically, the object recognition unit 124 accesses the database todetermine, regarding the own object detection apparatus 100 (the imagingregion 42), whether or not a result of the determination indicating thata search limitation will be imposed at the current timing is stored.

When it has been determined that there is no search limitation at thecurrent timing (NO in S114), the object recognition unit 124 performsobject recognition without imposing a search limitation. Therefore, theobject recognition unit 124 performs object recognition for the wholeimage acquired in S112 (Step S116). Then the object recognition unit 124generates object information, as described above. Then the objectrecognition unit 124 stores the generated object information in adatabase.

On the other hand, when it is determined that there is a searchlimitation at the current timing (YES in S114), the object recognitionunit 124 performs object recognition with imposing a search limitation.That is, the object recognition unit 124 extracts a search range fromthe database and performs object recognition for the extracted searchrange (Step S120). Then the object recognition unit 124 generates theobject information, as described above. Then the object recognition unit124 stores the generated object information in the database.

As described above, at a timing when the moving object is estimated toreach the own imaging region 42 (the imaging region of the second imagecapturing apparatus), the object recognition unit 124 performs objectrecognition of the moving object on the search range. Accordingly, atthe timing when the moving object is estimated to reach the own imagingregion 42, the search range is limited. It is therefore possible todetect a moving object more definitely.

As described above, the object information extraction unit 126 extractsobject information regarding a moving object that is the same as themoving object recognized by the object recognition processing performedby the object recognition unit 124 from the database (Step S122). Thenthe reliability calculation unit 128 calculates the integratedreliability regarding the recognized moving object using the objectinformation extracted by the object information extraction unit 126, asdescribed above (Step S124). Specifically, the reliability calculationunit 128 calculates, for each type, the reliability (integratedreliability) of the moving object using the type and the reliability ofthe moving object obtained as a result of object recognition performedby the object recognition unit 124 and the type and the reliability ofthe moving object included in the movement information.

As described above, the reliability calculation unit 128 re-calculates(updates) a reliability, whereby the accuracy of the reliability can beimproved. That is, the amount of data regarding the reliabilitycalculated using a plurality of pieces of object information is largerthan that in the reliability obtained in the object recognitionprocessing performed by the object recognition unit 124. Therefore, theaccuracy of the reliability calculated using the plurality of pieces ofobject information may become higher than the accuracy of thereliability obtained in the object recognition processing performed bythe object recognition unit 124.

Further, the reliability calculation unit 128 is configured to calculatethe reliability (integrated reliability) by calculating, for each type,the average of the reliability of the moving object obtained as a resultof object recognition performed by the object recognition unit 124 andthe reliability of the moving object included in the movementinformation. Accordingly, a reliability in which a plurality ofreliabilities are taken into account is calculated for each determinedtype of the moving object. Therefore, the reliability calculation unit128 is able to calculate the integrated reliability more appropriately.

As described above, the movement information generation unit 130generates movement information (Step S126). When the integratedreliability has been calculated in S124, the movement informationgeneration unit 130 generates the movement information in such a waythat the reliability of the corresponding moving object in the objectinformation becomes the integrated reliability. The movement informationtransmission unit 132 transmits the movement information generated inthe processing of S126 (Step S128). In this case, as described above,the movement information acquisition unit 112 may store the movementinformation in the database of the management server 50. Further, themovement information transmission unit 132 may transmit the movementinformation to the adjacent object detection apparatus 100.

FIG. 11 is a diagram for describing a specific example of processing inthe object detection system 1 according to the first embodiment. FIG. 11illustrates processing in a case in which the moving object S moves fromthe imaging region 42A to the imaging region 42B, as illustrated in FIG.3 . It is assumed that the moving object S has been recognized for thefirst time in the object detection apparatus 100A.

Since the moving object S is recognized for the first time in the objectdetection apparatus 100A, at this stage, no object detection apparatus100 generates movement information including object informationregarding the moving object S. Therefore, the object recognition unit124 of the object detection apparatus 100A performs object recognitionprocessing on the whole image captured by the image capturing apparatus101A without performing a search limitation, thereby recognizing themoving object S (S116).

Then, the movement information generation unit 130 of the objectdetection apparatus 100A generates movement information Ia includingobject information Isa of the moving object S (S126). It is assumed thatthe object ID of the moving object S is Sa in the object informationIsa. It is further assumed that the final recognition time of the movingobject S is t1 in the object information Isa. It is further assumed thatthe final recognition orientation of the moving object S is “a directionof the imaging region B”. Further, the final recognition speed of themoving object S is denoted by v1. Further, the feature amount of themoving object S is denoted by Va.

Further, it is assumed that the type and the reliability of the movingobject S are “heavy truck: 0.6” and “bus: 0.4”. Therefore, in the resultof the recognition by the object detection apparatus 100A, thereliability of the type (category) of the moving object S is higher in“heavy truck” than that in “bus”. That is, in the result of therecognition by the object detection apparatus 100A, the moving object Sis highly likely to be a “heavy truck”.

On the other hand, the object detection apparatus 100B that correspondsto the imaging region 42B toward which the moving object S travelsacquires the movement information Isa. Then the search limitationdetermination unit 114 of the object detection apparatus 100B determinesthat the moving object S is traveling toward the imaging region 42B fromthe final recognition orientation of the moving object S “the directionof the imaging region B”. Then the search limitation determination unit114 of the object detection apparatus 100B estimates, from the finalrecognition time t1 and the final recognition speed v1 of the movingobject S, that the moving object S will reach the imaging region 42B attime t2. That is, the search limitation determination unit 114 of theobject detection apparatus 100B determines that the moving objectappears in the angle of view of the image capturing apparatus 101B attime t2. Therefore, the search limitation determination unit 114 of theobject detection apparatus 100B determines that a search limitation willbe imposed at time t2 (S104). Then, the search range determination unit116 of the object detection apparatus 100B sets a predetermined range inthe direction of the imaging region 42A as a limited search range 60, asillustrated in FIG. 7 (S106).

When the image captured by the image capturing apparatus 101B isacquired at time t2, the object recognition unit 124 of the objectdetection apparatus 100B performs object recognition for the limitedsearch range (S120). Then the object recognition unit 124 of the objectdetection apparatus 100B detects the moving object S and generates theobject information Isb regarding the moving object S.

It is assumed, in the object information Isb, that the object ID of themoving object S is Sb. It is further assumed that the type and thereliability of the moving object S are “heavy truck: 0.3” and “bus:0.7”. Therefore, in the result of the recognition by the objectdetection apparatus 100B, the reliability of the type (category) of themoving object S is higher in “bus” than that in “heavy truck”. That is,in the result of the recognition by the object detection apparatus 100B,the moving object S is highly likely to be a “bus”. In this way, theresult of the recognition by the object detection apparatus 100B may bedifferent from the result of the recognition by the object detectionapparatus 100A. This is because the accuracy of the recognition may varydepending on an environmental factor such as the position where theimage capturing apparatus 101 is installed with respect to theintersection 40, visibility at the intersection 40, brightness in theintersection 40 depending on weather or the like, and the performance ofthe image capturing apparatus 101.

Further, the object information extraction unit 126 of the objectdetection apparatus 100B determines that the feature amount Va in theobject information Isa is similar to the feature amount of the movingobject S obtained in the object recognition processing in S120.Therefore, the object information extraction unit 126 of the objectdetection apparatus 100B extracts the object information Isa thatcorresponds to the moving object S (object ID: Sb) (S122).

Then the reliability calculation unit 128 of the object detectionapparatus 100B calculates the integrated reliability of the movingobject S using the extracted object information Isa (S124). In thiscase, the reliability calculation unit 128 of the object detectionapparatus 100B calculates, regarding the type “heavy truck”, that aharmonic mean of the reliability “0.6” in the object information Isa andthe reliability “0.3” in the object information Isb is “0.4”. Thereliability calculation unit 128 of the object detection apparatus 100Bfurther calculates, regarding the type “bus”, that a harmonic mean ofthe reliability “0.4” in the object information Isa and the reliability“0.7” in the object information Isb is “0.5”.

As described above, the accuracy of the recognition may vary dependingon an environmental factor regarding the image capturing apparatus 101.Therefore, a result of the recognition by one object detection apparatus100 is not always correct. As described above, in the example shown inFIG. 11 , in the result of the recognition by the object detectionapparatus 100A, the moving object S is highly likely to be a “heavytruck”, whereas in the result of the recognition by the object detectionapparatus 100B, the moving object S is highly likely to be a “bus”. Onthe other hand, according to this embodiment, the accuracy of thereliability can be increased by calculating the integrated reliabilityusing the reliability in the plurality of pieces of object information.

Further, when the moving object S moves in the imaging region 42C andthe imaging region 42D, object recognition is performed regarding themoving object S in both the object detection apparatus 100C and theobject detection apparatus 100D. Therefore, in both the object detectionapparatus 100C and the object detection apparatus 100D, data of thereliability regarding the moving object S is further obtained.Therefore, by calculating the integrated reliability further using thereliability obtained in object recognition by the object detectionapparatus 100C and the object detection apparatus 100D, the accuracy ofrecognition regarding the moving object S can be further increased. Thatis, when the integrated reliability is calculated using a large amountof data regarding reliability, a value of the integrated reliabilityregarding the correct type of the moving object S may tend to becomelarge. Therefore, according to this embodiment, the larger the number ofintersections 40 (the imaging regions 42) that the moving object Spasses through becomes, the higher the accuracy of recognition of themoving object S becomes. It is therefore possible to prevent the movingobject from being falsely recognized. For example, in the objectrecognition by the object detection apparatus 100A, the moving object S,which is actually a bus, may be falsely recognized to be a heavy truck.On the other hand, by calculating the integrated reliability, thereliability (the integrated reliability) of the bus may become high,whereby it is possible to prevent the moving object S from being falselyrecognized as a heavy truck.

MODIFIED EXAMPLES

Note that the present disclosure is not limited to the above-describedembodiments and may be changed as appropriate without departing from thespirit of the present disclosure. For example, the order of each step inthe flowcharts shown in FIGS. 9 and 10 can be changed as appropriate.Further, one or more steps of the flowcharts shown in FIGS. 9 and 10 maybe omitted. For example, in FIG. 10 , the processing of S122 and S124may be omitted.

Further, while the object detection method shown in FIGS. 9 and 10 isexecuted in the information processing apparatus 110 of the objectdetection apparatus 100 in the above-described embodiment, thisconfiguration is merely an example. For example, one or more of theprocesses shown in FIGS. 9 and 10 may be executed by the managementserver 50. For example, the processing of FIG. 9 may be executed by themanagement server 50. Further, all the processes in FIGS. 9 and 10 maybe executed by the management server 50. In this case, the managementserver 50 may receive images captured by the image capturing apparatus101 from each of the object detection apparatuses 100 and perform theaforementioned processing for each of the object detection apparatuses100. However, if all the processes are executed in the management server50, it is possible that communication loads may increase. Therefore,some of the processes in the object detection method may be preferablyexecuted by the object detection apparatus 100. For example, theprocessing regarding the object recognition may be preferably executedin the object detection apparatus 100.

Further, while the object recognition unit 124 detects an arbitrarymoving object, this configuration is merely an example. The objectrecognition unit 124 may recognize only a specific moving object such asa vehicle violating traffic regulations. Further, the image capturingapparatus 101 may not be fixed near the intersection 40. For example,images of the intersection 40 may be captured by an image capturingapparatus 101 mounted on a drone or the like.

Further, in the above-described examples, the program can be stored andprovided to a computer using any type of non-transitory computerreadable media. Non-transitory computer readable media include any typeof tangible storage media. Examples of non-transitory computer readablemedia include magnetic storage media (such as flexible disks, magnetictapes, hard disk drives, etc.), optical magnetic storage media (e.g.,magneto-optical disks), CD-Read Only Memory (CD-ROM), CD-R, CD-R/W, andsemiconductor memories (such as mask ROM, Programmable ROM (PROM),Erasable PROM (EPROM), flash ROM, Random Access Memory (RAM), etc.). Theprogram may be provided to a computer using any type of transitorycomputer readable media. Examples of transitory computer readable mediainclude electric signals, optical signals, and electromagnetic waves.Transitory computer readable media can provide the program to a computervia a wired communication line (e.g., electric wires, and opticalfibers) or a wireless communication line.

From the disclosure thus described, it will be obvious that theembodiments of the disclosure may be varied in many ways. Suchvariations are not to be regarded as a departure from the spirit andscope of the disclosure, and all such modifications as would be obviousto one skilled in the art are intended for inclusion within the scope ofthe following claims.

What is claimed is:
 1. An object detection system comprising: a centralprocessing unit (CPU) configured to: acquire movement information, whichis information regarding a moving object included in a first imagecaptured by a first image capturing apparatus among a plurality of imagecapturing apparatuses arranged in positions different from one another;determine a limited search range in a second image captured by a secondimage capturing apparatus among the plurality of image capturingapparatuses in accordance with the movement information; performrecognition of the moving object for the limited search range in thesecond image; and calculate, for each type of the moving object, areliability of recognition of the moving object using a type and areliability of the moving object obtained as a result of performingrecognition of the moving object and a type and a reliability of themoving object included in the movement information, wherein thereliability is calculated by calculating an average of the reliabilityof the moving object obtained as a result of performing recognition ofthe moving object and the reliability of the moving object included inthe movement information.
 2. The object detection system according toclaim 1, wherein the CPU is configured to determine the limited searchrange when the movement information indicates that the moving object istraveling toward an imaging region of the second image capturingapparatus.
 3. The object detection system according to claim 2, whereinthe CPU is configured to determine a predetermined range in the secondimage that corresponds to a direction of an imaging region of the firstimage capturing apparatus to be the limited search range.
 4. The objectdetection system according to claim 1, wherein the CPU is configured toperform recognition of the moving object for the limited search range ata timing when the moving object is estimated to reach an imaging regionof the second image capturing apparatus.
 5. The object detection systemaccording to claim 1, further comprising the plurality of imagecapturing apparatuses.
 6. An object detection method comprising:acquiring movement information, which is information regarding a movingobject included in a first image captured by a first image capturingapparatus among a plurality of image capturing apparatuses arranged inpositions different from one another; determining a limited search rangein a second image captured by a second image capturing apparatus amongthe plurality of image capturing apparatuses in accordance with themovement information; performing recognition of the moving object forthe limited search range in the second image; and calculating, for eachtype of the moving object, a reliability of recognition of the movingobject using a type and a reliability of the moving object obtained as aresult of performing recognition of the moving object and a type and areliability of the moving object included in the movement information,wherein the reliability is calculated by calculating an average of thereliability of the moving object obtained as a result of performingrecognition of the moving object and the reliability of the movingobject included in the movement information.
 7. A non-transitorycomputer readable medium storing a program for causing a computer toexecute: acquiring movement information, which is information regardinga moving object included in a first image captured by a first imagecapturing apparatus among a plurality of image capturing apparatusesarranged in positions different from one another; determining a limitedsearch range in a second image captured by a second image capturingapparatus among the plurality of image capturing apparatuses inaccordance with the movement information; performing recognition of themoving object for the limited search range in the second image; andcalculating, for each type of the moving object, a reliability ofrecognition of the moving object using a type and a reliability of themoving object obtained as a result of performing recognition of themoving object and a type and a reliability of the moving object includedin the movement information, wherein the reliability is calculated bycalculating an average of the reliability of the moving object obtainedas a result of performing recognition of the moving object and thereliability of the moving object included in the movement information.